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ABSTRACT 

This  paper  introduces  a  new  class  of  texture  features 
based  on  the  joint  occurrences  of  gray  levels  at  points 
defined  relative  to  edge  maxima.  These  features  are 
compared  with  previous  types  of  cooccurrence-based  features, 
and  experimental  results  are  presented  indicating  that  the 
new  features  should  be  useful  for  texture  classification. 
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1.  Introduction 


- Features  based  on  the  joint  frequencies  of  occurrence  of 

pairs  of  gray  levels  at  given  separations  are  often  used  for 
texture  analysis  Recently ,  Davis  et  al.  have  suggested 

w  using  joint  occurrences  of  local  maxima  of  a  local  property  (e.g., 

edge  maxima)  to  define  texture  features.  This  paper  proposes  a 
hybrid  approach  using  joint  gray  level  occurrences  at  and  near 
edge  maxima, 

Section  2  reviews  the  standard  cooccurrence  approach  and 
Davis’  approach,  and  also  describes  the  proposed  new  approach. 
Section  3  presents  several  sets  of  experimental  results,  using 
the  pictures  in  Figure  7,  which  indicate  that  the  proposed 
features  should  be  useful  for  texture  classification. 
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2.  Cooccurrence  matrices 


2.1  Gray  level  cooccurrence 

Given  a  vector  6= (Ax, Ay)  and  a  picture,  we  can  estimate 
the  joint  probability  density  of  the  pairs  of  gray  levels  that 
occur  at  pairs  of  points  separated  by  6  by  counting  all  such 
pairs  of  gray  levels  in  the  picture.  If  the  picture  has  been 
quantized  to  m  levels,  the  result  is  an  m  by  m  matrix,  where 
entry  (i,j)  is  the  number  of  times  gray  levels  i  and  j  occur 
at  separation  6.  In  our  experiments  we  will  use  the  symmetric 
form  of  this  matrix,  i.e. ,  we  define  the  entries  of  the  matrix 
to  be  the  numbers  of  times  that  pairs  of  gray  levels  occur  at 
separation  either  6  or  -6. 

When  6  is  small  compared  to  the  texture  element  size,  the 
high  values  in  Mg  are  concentrated  near  the  main  diagonal.  As 
6  approaches  texture  element  size,  the  gray  levels  of  points  will 
in  general  be  quite  different,  so  values  will  be  more  uniformly 
spread  out  in  .  For  example,  Figure  2  shows  the  coocccurrence 
matrices  derived  from  the  pictures  in  Figure  1  for  6= (0,2). 


2.2  Edge  maxima  cooccurrence 


Alternatively  to  measuring  cooccurrence  of  pairs  of  gray 
levels,  it  has  been  suggested  [3]  that  cooccurrence  of  other 
local  properties,  e.g.,  edge  orientations  at  pairs  of  points 
with  local  maximum  edge  magnitudes,  are  useful  texture  measures. 
This  class  of  cooccurrence  matrices  is  computed  as  follows: 
first,  a  set  of  feature  points  is  determined  by  applying  non¬ 
maximum  suppression  to  the  output  of  a  local  operator  on  a 
picture.  Next,  a  neighbor  function  is  used  to  pair  up  these 
local  maxima  based  on  their  relative  locations  and  descriptions. 
Finally,  the  cooccurrence  matrix  G  is  computed  by  counting  the 
pairs  of  descriptions  associated  with  these  pairs  of  points. 

That  is,  G(i,j)  is  the  number  of  pairs  that  have  descriptions 

1  and  j.  If  there  are  m  possible  descriptions  associated  with 
these  local  maxima,  then  G  is  m  by  m. 

In  this  study  we  used  as  our  local  operator  the  magnitude 
of  an  edge  detector  based  on  differences  of  averages  over  2  by 

2  neighborhoods,  and  we  used  edge  orientation  as  the  descriptor. 

A  point's  edge  magnitude  is  defined  to  be  the  maximum  absolute 
difference  of  2  by  2  blocks  of  pixels  oriented  in  directions  0°, 
45°,  90",  and  135°;  thus  G  is  a  4  by  4  matrix.  Non-maximum  sup¬ 
pression  was  then  applied  in  the  direction  normal  to  the  direction 
of  maximum  edge  response.  Figure  3  shows  the  edge  maxima 
computed  from  the  textures  in  Figure  1. 


Two  neighbor  functions  were  used  here;  they  are  similar  to 


those  in  [3].  The  first  specifies  a  set  of  points,  ,  in  an 
hourglass-shaped  region  centered  at  an  edge  maximum  and  oriented 
in  the  direction  of  the  edge  at  that  point;  each  triangular 
sector  has  angular  width  45°  and  height  5.  This  set  of  points 
is  shown  in  Figure  4a,  where  points  labeled  i  are  neighbors  of 
the  center  point  when  its  edge  orientation  is  45i.  (i,j)  is 

the  number  of  times  that  an  edge  maximum  with  orientation  j  is 
a  neighbor  of  an  edge  maximum  with  orientation  i.  We  call  GN 
the  edge  maxima  cooccurrence  matrix  for  all  neighbors  along  an 
edge. 


The  second  neighbor  function  is  also  an  hourglass-shaped 

region,  N2,  of  the  same  dimensions,  but  oriented  in  the  direction 

perpendicular  to  the  edge  (Figure  4b).  In  this  case,  an  edge 

maximum  p  is  paired  with  each  edge  maximum  q  that  occurs  in  the 

hourglass  area  centered  at  p  and  oriented  across  p's  edge.  Thus 

Gm  (i,j)  is  the  number  of  times  that  the  edge  maximum  with  orient- 
n2 

ation  j  is  a  neighbor  of  one  with  orientation  i.  We  call  GM 

n2 

the  edge  maxima  cooccurrence  matrix  for  all  neighbors  across  an 
edge. 

Measuring  cooccurrence  of  edge  orientations  of  a  point  and 
its  neighbors  in  the  edge  direction  should  indicate  properties 
of  the  curvature  of  texture  element  boundaries;  if  texture  bound¬ 
aries  are  smooth,  then  the  values  in  GM  should  be  concentrated 


near  the  main  diagonal.  Conversely,  for  a  fine  texture  or  a 

jagged-border  coarse  texture,  maxima  will  not  in  general  have 

the  same  edge  orientation  as  their  paired  points,  so  the  values 

in  G„  will  be  more  spread  out.  Figure  5a  shows  these  cooccur- 
N1 

rence  matrices  for  the  textures  in  Figure  1.  The  cooccurrence 
matrix  produced  by  pairing  edge  maxima  across  the  edges  should 
also  measure  shape  and  density  properties  of  texture  elements. 
Figure  5b  shows  G„  matrices  for  the  textures  in  Figure  1. 


2.3  Gray  level  cooccurrence  based  on  edge  maxima 

Gray  level  cooccurrence  uniformly  measures  a  fixed  spatial 
relationship  of  gray-tones  and  therefore  its  usefulness  depends 
on  the  adequacy  of  this  "average"  gray  level  dependence  over  all 
parts  of  a  texture.  Edge  maxima  cooccurrence  measures  edge 
orientation  dependence,  which  captures  texture  element  boundary 
properties,  but  ignores  the  tonal  properties  of  these  elements. 

We  now  define  a  new  class  of  methods  which  measures  the  gray  level 
cooccurrence  of  pairs  of  points  at  and  near  edge  maxima.  This 
hybrid  scheme  attempts  to  localize  the  measure  of  gray  level 
spatial  dependency  at  selected  points  of  a  texture  and  in  selected 
directions.  Features  derived  from  these  matrices  will  indicate 
properties  of  pairs  of  points  with  specified  relationships  to 
the  locations  and  orientations  of  the  texture's  edge  maxima. 

As  in  edge  maxima  cooccurrence,  an  edge  map  is  first  computed 
by  applying  an  edge  detector  and  then  suppressing  non-maxima. 

For  each  edge  orientation,  a  neighbor  function  is  given  which 
pairs  points  at  and  near  an  edge  point.  The  cooccurrence  matrix 
H  gives  the  gray  level  cooccurrences  of  these  pairs  of  points , 
where  H(i,j)  is  the  number  of  pairs  of  gray  levels  that  occur 
at  a  specified  displacement  relative  to  the  locations  and  orient¬ 
ations'  of  edge  maxima. 

The  edge  detector  used  here  is  the  same  one  described  in 
Section  2.2,  i.e.,  differences  of  2  by  2  averages  in  four  direct¬ 
ions.  We  now  define  several  neighbor  functions  and  the  resulting 


matrices  computed  using  them  which  may  be  useful  in  estimating 
joint  gray  level  density  near  texture  element  boundaries.  Let 
p  be  a  point  having  non-zero  edge  magnitude  and  edge  orientation 
0,  and  let  d  be  a  given  distance. 

a)  Most  similar  neighbor  along  an  edge. 

Let  q  be  the  point  at  distance  d  from  p  in  the  9  direction 

and  let  r  be  the  point  at  distance  d  from  p  in  direction  0+if . 

Let  i,j/k  be  the  gray  levels  of  points  p,q,r,  respectively. 

Increment  H_  (i,j)  if  |i-j|  =min ( | i-j | , | i-k | ) ;  otherwise, 

F1 

increment  H„  (i,k).  Thus,  H  measures  the  gray  level  cooccur- 
1  F1 

rences  of  edge  points  with  their  most  similar  (in  terms  of  gray 

level)  neighbors  in  the  directions  along  their  edges. 

This  matrix  measures  properties  of  the  curvature  of  texture 

element  boundaries.  If  boundaries  are  generally  straight,  then 

pairs  of  points  will  usually  belong  to  the  same  population  and 

high  values  will  occur  along  the  main  diagonal  of  H  .  If  the 

F1 

boundary  turns  frequently,  then  edge  points  will  often  be  paired 
with  points  inside  or  outside  a  texture  element  depending  on  the 
direction  of  curvature.  Figure  6a  illustrates  this  definition 
for  d=2  for  the  textures  in  Figure  1. 

b)  Most  similar  neighbor  across  an  edge. 


Let  q  be  the  point  at  distance  d  from  p  m  direction 
0+(ir/2)  and  let  r  be  the  point  at  distance  d  from  p  in  the 
0-(ir/2)  direction.  Increment  H_  (i,j)  if  I i- j I =min( | i-j | , | i-k | ) 


otherwise  increment  H_  (i,k).  This  method  pairs  each  edge  point 

f2 

with  that  point  which  is  perpendicular  to  the  edge's  direction 
and  has  the  most  similar  gray  level  to  that  of  the  given  point. 
Thus  each  point  is  paired  with  a  neighbor  which  is  (presumably) 
inside  the  same  texture  element;  features  based  on  this  matrix 
will  reflect  intra-region  joint  gray  level  probabilities  of  a 
texture's  constituent  elements.  Figure  6b  shows  the  method  for 
d=2. 

c)  Least  similar  neighbor  across  an  edge. 

This  method  is  the  same  as  (b)  except  that  we  increment 

H-,  (i#j)  if  |  i-j  |=max(  |  i-j  I ,  I  i-k  | )  ,  and  otherwise  increment 
f3 

H_  (i,k).  This  operation  pairs  points  which  are  on  opposite 
*  3 

sides  of  an  edge  and  therefore  should  measure  joint  gray  level 
probability  densities  for  adjacent  texture  elements.  See  Figure 


^  d)  Pair  of  neighbors  across  an  edge. 

This  is  also  the  same  as  (b) ,  except  that  if  q  is  the 

most  similar  point  to  p,  then  we  increment  H„  (j,£),  where  l  is 

F4 

the  gray  level  of  the  point  s  which  is  at  distance  d  from  q  in 

direction  0+(ir/2).  Similarly,  if  r  is  the  most  similar  point 

to  p,  then  we  increment  H„  (j,m),  where  m  is  the  gray  level  of 

f4 

the  point  t  which  is  at  distance  d  from  r  in  direction  6—  (tt/2 )  . 


Here,  an  edge  point  determines  the  direction  of  its  interior  and 
then  continues  in  that  direction  to  find  another  point.  Thus 


3.  Experiments 


A  pilot  study  has  been  performed  for  preliminary  evaluation 
of  texture  discriminability  based  on  single  features  derived 
from  the  various  cooccurrence  matrices  defined  in  Section  2 . 
Subjective  clustering  criteria  were  used  to  evaluate  performance 
(this  is  reasonable  considering  the  size  and  dimensionality  of 
the  experiments) . 

Two  sets  of  texture  samples  were  used  in  the  experiments. 

The  first  is  a  set  of  terrain  samples  selected  from  a  LANDSAT 
image  of  eastern  Kentucky  and  was  used  in  [4] .  Three  repre¬ 
sentative  images  were  chosen  from  each  of  the  three  geological 
terrain  types,  as  shown  in  Figure  7a.  A  second  data  base  was 
chosen  from  the  Brodatz  collection  of  textures  [5] .  Three 
windows  of  each  of  four  texture  types  (the  same  types  used  in 
[6])  were  chosen  and  are  shown  in  Figure  7b.  Both  data  sets 
were  histogram  flattened  to  remove  any  effects  of  unequal  bright 
ness  and  contrast  in  the  originals.  (One  picture  from  each  set 
was  shown  in  Figure  1.) 

The  edge  detector  was,  as  mentioned  earlier,  the  absolute 
difference  of  averages  over  2  by  2  neighborhoods  oriented  in 
directions  0°,  45°,  90°,  and  135°.  Non-maximum  suppression  used 
a  1  by  4  neighborhood  centered  at  the  point  of  interest  and 
oriented  in  the  perpendicular  direction  to  the  edge.  The  output 
of  this  process  for  the  two  data  bases  is  shown  in  Figure  8. 


H,,  measures  the  joint  gray  level  probability  of  pairs  of 
F4 

interior  points.  See  Figure  6d. 

e)  All  neighbors  along  an  edge. 

Consider  again  the  hourglass-shaped  neighborhood  of 

points,  N^r  oriented  in  direction  0  at  p  as  specified  in  Section 

2.2.  For  each  point  q€N.  ,  increment  H_  (i, j) .  This  case  is 

l  f5 

similar  to  (a) ;  a  set  of  cooccurrences  is  computed  in  the  edge 
direction  at  each  edge  maximum.  Again  the  joint  probability 
desnity  should  be  influenced  by  the  shape  characteristics  of 
texture  elements.  Figure  6e  illustrates  this  method. 

f)  All  neighbors  across  an  edge. 


This  is  the  same  as  (e)  except  that  the  hourglass 

neighborhood  N2»  oriented  across  the  edge,  is  used.  Examples 

of  this  matrix,  H_  ,  are  shown  in  Figure  6f. 

F6 


Four  features,  originally  defined  by  Haralick  [7],  were 

computed  from  each  normalized  cooccurrence  matrix  (i.e.,  given 

matrix  M,  define  p ( i , j )  =  M(i, j)/EEM(k,£) ) : 

k  l 

1)  Contrast  (CON)  =  E  E (i- j ) 2p (i , j ) 

i  j 

2 

2)  Angular  second  moment  (ASM)  =  E  Ep(i,j) 

i  j 

3)  Entropy  (ENT)  =  -E  Ep(i,j)  log  p(i,j) 

i  j 

4)  Correlation  (COR)  =  E  E [i jp (i, j) -y  y  1 / (o  a  )  ,  where 

i  j  Y  Y 

ux  and  ox  are  the  mean  and  standard  deviation  of  the 
row  sums  of  the  cooccurrence  matrix,  and  yy  and  are 
analogous  statistics  of  the  column  sums. 

The  set  of  feature  values  derived  using  a  given  data  set, 
cooccurrence  matrix  method  and  feature  type,  were  plotted  along 
a  line  with  each  window's  value  uniquely  designated.  Subjective 
evaluation  of  a  feature  was  based  on  the  visual  separability  of 
the  texture  classes  in  this  plot. 

After  preliminary  experiments  using  different  values  of  6 
(distances  1,  2,  4  and  directions  0,  45,  90,  135),  it  was  found 
that  individual  features  based  on  5= (0,2)  did  at  least  as  well 
as  features  based  on  the  other  separations.  These  values  have 
been  plotted  in  Figures  9a  and  10a  for  the  Kentucky  and  Brodatz 

*  In  the  case  of  the  edge  maximum  cooccurrence  matrices ,  matrix 
indices  1,  2,  3,  4  correspond  to  edge  directions  90°,  0°,  45°, 
and  135°,  respectively. 


data  bases  respectively.  In  particular,  the  contrast  and  cor¬ 
relation  features  for  the  Kentucky  samples  are  seen  to  group 
the  three  classes  into  well  separable  clusters.  For  the  Brodatz 
textures,  none  of  the  features  were  very  successful. 

Edge  maxima  cooccurrence  based  features  were  computed  for 
the  two  methods  described  above,  namely  all  neighbors  across  an 
edge  and  all  neighbors  along  an  edge.  These  results  are  shown 
in  Figures  9b, c  and  10b, c.  These  features  did  not  do  well  in 
distinguishing  the  terrain  types,  but  the  correlation  feature 
for  neighbors  across  an  edge  separated  the  four  Brodatz  textures. 

A  partial  explanation  of  these  results  can  be  derived  by  examining 
the  edge  maxima  shown  in  Figure  8.  This  class  of  methods  depends 
on  the  reliability  and  distinguishability  of  these  thinned  edge 
maps;  if  the  edge  maps  associated  with  different  features  are 
not  very  different,  then  we  would  expect  features  based  on  the 
methods  to  be  less  successful.  Here,  the  terrain  edge  maps  are 
not  very  distinguishable  and  produce  poor  results,  while  the 
Brodatz  texture  edge  maps  can  be  visually  discriminated. 

The  same  features  were  computed  for  each  of  the  six  methods 
for  measuring  gray  level  cooccurrence  near  edge  maxima.  Distance 
d=2  was  used  in  each  case.  Figures  9d-i  and  lOd-i  show  the 
plotted  values .  The  results  on  the  Kentucky  data  were  uniformly 
poor  in  separating  the  three  terrain  classes.  Using  the  Brodatz 
samples,  however,  good  separability  was  obtained  using  the  entropy 
feature  on  each  of  the  six  matrices.  This  class  of  methods  also 


suffers  when  the  thinned  edge  maps  are  not  sufficiently  different 
to  distinguish  the  textures;  in  these  cases  it  seems  that  other 
information  than  that  found  hear  edges  is  needed.  In  addition, 
these  methods  are  based  on  the  assumption  that  textures  are 
describable  as  collections  of  primitive  elements,  so  that  edge 
maxima  will  correspond  to  points  on  the  boundaries  of  these 
elements.  The  Brodatz  textures  fit  this  model  and  the  results 
are  successful;  the  terrain  textures  are  wrinkled  and  contain 
line-like  elements  and  consequently  the  methods  used  here  are 
not  the  most  appropriate  ones. 

Tables  1  and  2  summarize  the  results  of  Figures  9  and  10, 


respectively 


Feature 


Cooccurrence 

Type 

Method 

ASM 

CON 

ENT 

COR 

Gray  level 

6  =  (0,2) 

B/A,C 

A/B/C 

B/A,C 

A/B/C 

Along  edge 

poor 

poor 

poor 

A/B 

Edge  maxima 

Across  edge 

poor 

poor 

poor 

A/B 

Most  similar 
neighbor  along 
edge 

poor 

poor 

poor 

poor 

Most  similar 

neighbor  across 
edge 

poor 

B/C 

poor 

poor 

Gray  level 

Least  similar 

at  and  near 
edge  maxima 

neighbor  across 
edge 

poor 

B/C 

poor 

poor 

Pair  of  neighbors 
across  edge 

poor 

poor 

poor 

C/A,] 

All  neighbors 
along  edge 

poor 

poor 

poor 

A/B 

All  neighbors 
across  edge 

poor 

poor 

poor 

poor 

Table  1.  Summary  of  Figure  9  plots  describing  the 

separability  of  the  terrain  textures.  A,B,c 
denote  the  three  terrain  classes;  slash 
means  "separated". 


I 


4.  Conclusion 

A  new  class  of  cooccurrence  matrices  has  been  defined  which 
measures  the  joint  occurrences  of  gray  levels  at  pairs  of  points 
at  locations  and  separations  defined  relative  to  the  positions 
and  orientations  of  edge  maxima.  This selective  cooccurrence 
approach  is  not  likely  to  be  sensitive  to  the  size  of  texture 
elements  as  is  the  standard  gray  level  cooccurrence  method. 
Experimental  results  on  a  small  number  of  coarse  textures,  though 
not  statistically  significant,  appear  to  show  a  marked  improvement 
in  the  features  based  on  gray  level  cooccurrence  near  edge  maxima 
over  features  based  on  fixed  separation  gray  level  cooccurrence. 


Feature 


Cooccurrence 

Type 

Method 

ASM 

CON 

ENT 

COR 

Gray  level 

6  =  (0,2) 

W/G,R,S 

G/W/R,  S 

G/W/R,  S 

G/W/R,  S 

Along  edge 

W/R,  S 

R/S/G , W 

W/G,  R,  S 

G/R/S , W 

Edge  maxima 

Across  edge 

poor 

R/W/G ,  S 

W/G,S 

R/G , S , W 

Most  similar 
neighbor  along 
edge 

R/G 

poor 

G/R/S/W 

R/G,  S 

Most  similar 
neighbor  across 
edge 

R/G 

poor 

G/R/S/W 

R/S,  W 

Gray  level 
at  and  near 
edge  maxima 

Least  similar 
neighbor  across 
edge 

R/G 

G,  S/R,  W 

G/R/S/W 

G/R/W 

Pair  of  neighbors 
across  edge 

G, S/R, W 

W/G,  S 

G/R/S/W 

R/G,S,W 

All  neighbors 
along  edge 

G/R/W 

W/G,  S 

G/R/S/W 

G,  S/R,W 

All  neighbors 
across  edge 

G/R 

G/R,  W 

G/R/S/W 

G/R,  W 

Table  2.  Summary  of  Figure  10  plots  describing  the 
separability  of  the  Brodatz  textures.  G  = 
grass,  R  =  raffia,  S  =  sand,  W  =  wool;  slash 
means  "separated". 
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Figure  1.  A  LANDSAT  terrain  texture  and  the 
raffia  texture  from  Brodatz  [5] . 


Figure  2.  Gray  level  cooccurrence  matrices  M,q  ^ ) 
for  the  textures  in  Figure  1 .  ' 

Each  entry's  value  was  scaled  by  ln32 
and  displayed  as  a  2  by  2  block  for 
visibility . 


Figure  3. 


Edge  maxima  after  non-maximum  suppression 
for  pictures  in  Figure  1.  (Threshclded 
for  visibility.) 


Gray  level  cooccurrence  matrices  based  on  edge 
maxima  for  textures  in  Figure  1.  a-f  correspond 
to  matrices  Hp-^-Hpg.  Each  entry's  value  was 
scaled  by  ln32. 
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Figure  7.  Data  bases  after  histogram  flattening. 

(a)  LANDSAT  terrain  samples.  Top  row  is 
Mississippian  limestone  and  shale,  second 
row  is  Pennsylvanian  sandstone  and  shale, 
and  third  row  is  Lower  Pennsylvanian  shale. 

(b)  Brodatz  textures.  Top  row  is  sand  (plate 
D29 ) ,  second  row  is  grass  (L9) ,  third  row  is 
wool  (D19) ,  and  fourth  row  is  raffia  (D84) . 


Figure  8. 


Edge  maxima  after  non-maximum  suppression  for 
textures  in  Figure  7.  (Thresholded  for  visibility.) 
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Figure  9d.  Features  derived  from  H_ 


for  terrain  samples 
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Figure  9f.  Features  derived  from  H_.  for  terrain  samples 
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Figure  9i.  Features  derived  from  HP  for  terrain  samples 
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Figure  10c.  Features  derived  from  G.t  for  Brodatz  textures 
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Figure  lOg.  Feature*  derived  from  H_  for  Brodatz  textures 
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